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DESCRIPTION 
FIGURE READING APPARATUS, METHOD AND PROGRAM 

5 Technical Field 

[0001] The present invention relates to a figure reading apparatus which 
conducts pattern recognition on characters or the like represented by a 
color image. 

10 Background Art 

[0002] There is a conventional technique of recognizing figures such as 
characters represented by a color image without being affected by the 
illumination condition or the hke. In this technique, it is made possible to 
recognize a subject figure with high precision by using combinations of 

15 main color components of the subject figure in order to cope with 

separation caused in a figure area of a recognition subject by an influence 
of, for example, a shadow depending upon the illumination condition or a 
coated surface thinned by deterioration or the like condition. 
[0003] FIG. 1 shows an example of a system for removing the influence of 

20 the illumination condition or the Hke in the conventional figure reading. 

The conventional figure reading system includes image input means, main 
color extraction means, combination generation means, figure area 
extraction means and figure recognition means. 

[0004] The conventional system having such a configuration operates as 
25 described hereafter. With respect to a color image input fi-om the image 
input unit, the main color extraction means extracts main colors on the 
basis of a local peak value in a color histogram included in the color image, 
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and resolves the color image into main color images respectively of the 
extracted main colors. The combination generation means combines the 
main color images generated by the resolving and generates combined 
images. The figure area extraction means extracts character area 
5 candidates firom the main color images and the combined images. The 
figure recognition means conducts character recognition processing on aU 
extracted character area candidates, and obtains the best character reading 
result on the basis of character recognition firequencies in a result of the 
character recognition processing. 

10 [0005] Furthermore, there is a pattern recognition apparatus for selecting 
a main color having a distance firom a main color which is a certain 
threshold or less firom among the main colors in the RGB color space in the 
pattern recognition of characters or the like indicated in a color image (see, 
for example, Patent Docimient 1). 

15 Patent Document 1" JP 2003- 16444 A 

Disclosure of the Invention 

Problem to be Solved by the Invention 

[0006] In the conventional technique, however, all combinations of the 
20 main color components extracted fi:om the subject figure are used. 
Accordingly, there is a problem that the processing time increases 
remarkably as the main colors increase. 

[0007] In other words, when generating combined images, effective 
combinations cannot be determined. In the subsequent figore area 
25 extraction means and figure recognition means, therefore, it is necessary to 
conduct processing on extra candidates as weU. Accordingly, the general 
processing rate cannot be increased. 
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[0008] Therefore, an object of the present invention is to provide a figure 
reading apparatus, method and program capable of increasing the 
processing rate remarkably while maintaining the recognition precision in 
figure recognition or the like using a color image. 

Means for Solving the Problem 

[0009] In order to achieve the object, the present invention provides a 
figure reading apparatus for reading, recognizing and outputting a figure, 
the figure reading apparatus including image input means for inputting a 
color image, main color extraction means for extracting main colors 
contained in the color image and resolving the color image into main color 
images respectively of main colors extracted firom the input color image, 
projection means for projecting points of extracted main colors on a 
three-dimensional space onto a plane by using a conversion expression, 
candidate narrowing down means for removing combinations that need not 
be combined according to featxures of distance between main colors 
projected onto a two-dimensional space, and combination generation means 
for combining images firom candidates narrowed down and generating an 
image area candidate. 

[0010] Owing to the configuration above described, extra candidates are 
not generated, but improvement of the processing speed as a whole can be 
anticipated. Furthermore, it is possible to eUminate invaHd combinations 
when removing the influence of the iUumination condition or the hke. 



Effect of the Invention 

[0011] A first effect of the present invention is that effective extraction of 
candidate areas becomes possible in figure recognition processing affected 



by the illumination condition or the coated surface thinned by deterioration 
or the like condition. 

[0012] A second effect is that the processing rate can be made high while 
maintaining the precision of the whole processing, by conducting effective 
extraction of candidate areas. 

Brief Description of the Drawings 

[0013] FIG. 1 is a diagram showing an example of a conventional figure 
reading system; 

FIG. 2 is a diagram showing a schematic configuration of a figure 
reading apparatus according to the present invention and its processing 
procedure; 

FIG. 3 is a diagram showing a schematic hardware configuration of 
a figure reading apparatus; 

FIG. 4 is a diagram showing projection fi-om a three-dimensional 
space of colors onto a plane; 

FIG. 5 is a diagram showing a schematic configuration of a figure 
reading apparatus according to a third embodiment and its processing 
procedure. 

Description of Reference Numerals 
[0014] 10" Image input apparatus 

20- Data processing apparatus 

Best Mode for Carrying Out the Invention 

[0015] The best mode for carrying out the present invention will now be 
described with reference to the drawings. 



(First Embodiment) 

[0016] FIG. 2 shows a schematic configuration of a figure reading 
apparatus according to the present invention and its processing procedure. 
The present embodiment includes an image input apparatus 10 such as a 
digital camera or a scanner capable of acquiring a color image, and a data 
processing apparatus (computer) 20 capable of conducting processing on 
the color image input from the image input apparatus 10 under program 
control. 

[0017] FIG. 3 shows a schematic hardware configuration of the figure 
reading apparatus. In the data processing apparatus 20, a CPU 12, a 
ROM 13 and a RAM 14 are connected to a system bus 11. In addition, a 
magnetic disk apparatus 15 is connected to the system bus 11 via an 
input-output interface 15a. A cache memory is moimted on the CPU 12, 
and the CPU 12 conducts instruction execution and arithmetic operation 
processing. A BIOS (basic input/output system) and so on are stored in 
the ROM 13 as firmware. The RAM 14 is a main storage, and the RAM 14 
stores a program and data read out fi-om the magnetic disk apparatus 15. 
An operating system, an application program and so on are stored in the 
magnetic disk apparatus 15. A program, image data and so on for 
executing the present invention are also stored in the magnetic disk 
apparatus 15. 

[0018] An image input apparatus 10, a display 16 and a printer 17 are 
connected to the system bus 11 via an input interface 10a, a display 
interface 16a and a output interface 17a, respectively. In addition, the 
system bus 11 is connected to a network 19 via a communication interface 
18 to conduct data transmission and reception with a different server or 
terminal. 



[0019] The data processing apparatus 20 functions as main color 
component extraction means, projection means, candidate narrowing 
means and combined image generation means by using a control program. 
Operations of these means wiU now be described in brief. 
[0020] The main color component extraction means extracts main color 
components from a color image. The projection means presumes a 
direction in which color components in an area separated according to the 
sunshine conditions or the like are separated on the three-dimensional 
space of colors, and generates a projection image of each color component 
on a plane perpendicular to an approximate separation direction. For 
example, the RGB space is supposed. Under the influence of the sunshine 
conditions, there is a feature that a shift is conducted in the approximate 
direction of a vector (R, G, B) = (l, 1, 1). Therefore, a projection image of 
color components projected onto a plane ofR + G + B = 0 from the direction 
of (R, G, B) = (1, 1, 1) is found. The candidate narrowing down means 
compares points of the projected color components, and narrows down 
candidate combinations by using a threshold found by experiments 
beforehand. The combined image generation means determines main 
colors, and generates a combined image from candidates narrowed down. 
[0021] Operation in the present embodiment will now be described with 
reference to the flow chart shown in FIG. 2. First, an image input from 
the image input apparatus 10 is input to the data processing apparatus 20 
as a color image (step Si). With respect to the input color image, the main 
color component extraction means in the data processing apparatus 20 
extracts main colors on the basis of a local peak value in a color histogram 
contained in color information, and resolves the color image into main color 
images respectively of the extracted main colors (step S2). Subsequently, 



points of the main colors on the three-dimensional space extracted at the 
step S2 are projected onto the plane ofR + G + B = 0 from the direction of 
(R, G, B) = (1, 1, 1) as shown in FIG. 4 by using the following conversion 
expression (step S3). 
[0022] [Equation 1] 
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In the expression, (x' y' z' l) on the right side represents coordinates 
before conversion, and (x y z l) on the left side represents coordinates after 
conversion. 

10 [0023] In addition, distances between points of the main colors on the 
projection plane found at the step S3 are compared. If the distance is 
within a threshold range found by experiments beforehand, then the 
combination of the main colors is regarded as vahd. If the distance is 
outside the threshold range, then the combination is regarded as invaUd 

15 (step S4). As a result, an image area candidate combination is determined 
(steps S5 and S6). Subsequently, noise removal processing for removing 
portions other than the extraction target is conducted on the determined 
image area (step S7). Finally, the target subject is recognized by 
conducting figxu-e recognition processing (step S8). 
20 [0024] Efifects brought by the first embodiment will now be described. In 
the present embodiment, it becomes possible to remove meaningless extra 
combinations when determining a combined image, and combine only 
effective areas such as character areas separated imder the influence of the 
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illumination condition or the coated surface thinned by deterioration or the 
like condition. As a result, it becomes possible to decrease the number of 
candidate areas. In the subsequent noise removal processing and 
recognition processing, therefore, it can be anticipated to increase the 
processing rate while maintaining the recognition precision. 
[0025] Furthermore, since meaningless extra combinations are removed in 
the present embodiment, finally extra results are not output. Therefore, 
the present embodiment is effective in selection processing of final results 
as well. 

(Second Embodiment) 

[0026] A second embodiment of the present invention will now be 
described. A schematic configuration of a figure reading apparatus 
according to the present embodiment and a processing procedure in the 
apparatus are the same as those in the first embodiment shown in FIG. 2. 
[0027] The present embodiment differs fi-om the first embodiment in the 
projection function of the projection means. First, the main color 
component extraction means extracts main color components from a color 
image. Even if a subject image is exposed to light from a specific light 
source, the projection means generates a projection image on a plane 
perpendicular to a direction indicated by a color of the Ught source from the 
direction. The candidate narrowing down means compares points of the 
projected color components, and narrows down candidate combinations by 
using a threshold found by experiments beforehand. The combined image 
generation means determines main colors, and generates combined images 
firom candidates narrowed down. 

[0028] Operation in the present embodiment will now be described with 
reference to the flow chart shown in FIG. 2. First, an image input from 
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the image input apparatus 10 is input to the data processing apparatus 20 
as a color image (step Si). With respect to the input color image, the main 
color component extraction means in the data processing apparatus 20 
extracts main colors on the basis of a local peak value in a color histogram 
contained in color information, and resolves the color image into main color 
images respectively of the extracted main colors (step S2). 
[0029] Subsequently, points of the main colors on the three-dimensional 
space extracted at the step S2 are projected onto a plane. As for a subject 
affected by a specific light soiurce such as a color Ught soiirce, however, it is 
considered that color information is also changed in a specific direction 
other than the direction of (R, G, B) = (1, 1, 1) according to a feature of the 
light source. A projection plane (perpendicular to the change direction) is 
found on the basis of a feature of the change, and projection onto the plane 
is conducted (step S3). In addition, distances between points of the main 
colors on the projection plane found at the step S3 are compared. If the 
distance is within a threshold range found by experiments beforehand, 
then the combination of the main colors is regarded as valid. If the 
distance is outside the threshold range, then the combination is regarded 
as invalid (step S4). As a result, an image area candidate combination is 
determined (steps S5 and S6). Subsequently, noise removal processing for 
removing portions other than the extraction target is conducted on the 
determined image area (step S7). Finally, the target subject is recognized 
by conducting figure recognition processing (step S8). 

[0030] In the present embodiment, a projection image of a subject affected 
by a specific light source such as a color fight source can be generated onto 
a plane perpendicular to a direction indicated by a color of the Ught sovurce 
fi"om the direction. 



(Third Embodiment) 

[0031] A third embodiment of the present invention will now be described. 
In the present embodiment, the projection means in the first embodiment 
and the projection means in the second embodiment can be selected and 
executed. FIG. 5 shows a schematic configuration of a figure reading 
apparatus according to the present embodiment and a processing procedure 
in the apparatus. 

[0032] The data processing apparatus 20 functions as main color 
component extraction means, projection means A (the projection means 
according to the first embodiment) or projection means B (the projection 
means according to the second embodiment), candidate narrowing means 
and combined image generation means by using a control program. In the 
present embodiment, a candidate eirea is generated by conducting 
combination using distances between points of main color components 
projected using the projection means A or the projection means B. 
[0033] First, main color component extraction means extracts main color 
components firom a color image. Using a threshold found beforehand 
usmg experiments, a decision is made as to distances between main color 
components projected from a three-dimensional space onto a 
two-dimensional space by the projection means A or the projection means B. 
Narrowing down is conducted. Combined images (two combinations, three 
combinations, and n combinations) of a plurality of patterns of main color 
components are generated. 

[0034] Operation in the present embodiment will now be described with 
reference to the flow chart shown in FIG. 5. First, an image input firom 
the image input apparatus 10 is input to the data processing apparatus 20 
as a color image (step Sll). With respect to the input color image, the 
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main color component extraction means in the data processing apparatus 
20 extracts main colors on the basis of a local peak value in a color 
histogram contained in color information, and resolves the color image into 
main color images respectively of the extracted main colors (step S12). 
[0035] Subsequently, points of the main colors on the three-dimensional 
space extracted at the step S2 are projected onto a plane by using the 
projection means A or the projection means B (step S13 or S14). In 
addition, distances between n points of the main colors (two combinations, 
three combinations, and n combinations) on the projection plane foimd at 
the step S13 or S14 are compared. If the distance is within a threshold 
range found by experiments beforehand, then the combination of the main 
colors is regarded as vaUd. If the distance is outside the threshold range, 
then the combination is regarded as invalid (step S15). As a result, an 
image area candidate combination is determined (steps S16 and SI 7). 
Subsequently, noise removal processing for removing portions other than 
the extraction target is conducted on the determined image area (step S18). 
Finally, the target subject is recognized by conducting figure recognition 
processing (step 19). 

[0036] Effects of the third embodiment will now be described. In the 
present embodiment, it can be also coped with the case where the subject 
area is divided into a plurality of areas by using combinations of a pluraUty 
of patterns. As a result of combining a plurality of patterns, however, the 
processing time load is increased. Even under such a situation, however, 
it becomes possible to remove meaningless extra combinations when 
determining a combined image, and combine only effective areas such as 
character areas separated under the influence of the illumination condition 
or the coated surface thinned by deterioration or the like condition. As a 
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resxilt, it becomes possible to decrease the ntunber of candidate areas. In 
the subsequent noise removal processing and recognition processing, 
therefore, it can be anticipated to increase the processing rate while 
maintaining the recognition precision. 

5 

Industrial Applicability 

[0037] The figure reading apparatus according to the present invention 
can be applied to recognition processing utilizing a color image, in which 
processing on combination of color components is needed, for example, 
10 under the influence of the illumination condition. Furthermore, the figure 
reading apparatus according to the present invention can also be applied to 
recognition of an object having a plurahty of color areas. 
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